Lately, highly automated driving (HAD) has become more and more important and has received the attention of many players in the automotive industry as well as that of many scientists working on machine learning and robotics. An autonomous car (driverless car, self-driving car, robotic car) is a vehicle that is capable of sensing its environment and navigating without human input.
Autonomous cars can detect their surroundings using a variety of techniques and sensors such as radar, LIDAR, GPS, odometry, and computer vision. Advanced control systems interpret sensory information to identify appropriate navigation paths, as well as obstacles and relevant signage. Autonomous cars have control systems that are capable of analyzing sensory data to distinguish between different vehicles on the road, which is very useful in planning a path to the desired destination.
HAD applications are using various sensors, e.g. cameras, lidar and radar systems, to perceive the environment of the vehicle. Based on the information provided by these sensors, all kinds of dynamic road users, e.g. cars, pedestrians and bicycles, as well as static objects such as signs, road markings, etc. can be detected. To come to a highly reliable representation of the environment, the information from various sensors, e.g. lidars, cameras, and radars, needs to be combined or fused. The fused information leads to the environmental model which may be used as the main input for the decision-making process of a self-driving car. In addition, parts of this information will be shown to the driver to increase his or her trust in the capabilities of our self-driving car. Sensor fusion puts the information from the various sensors together and removes duplicates and wrong information while improving the quality of the correct information. Sensor fusion works on uncertain information represented by covariance matrices, and combines it into something more reliable, i.e. less uncertain, by using algorithms such as the Hungarian method and Kalman filter. By doing so, the quality of the information that is provided is improved and thus leads to less false positives and false negatives. A false positive might lead to an emergency braking although there is no reason for it, whereas a false negative might lead to an accident as a consequence of an object, such as another car, not being detected. Sensor fusion reduces the likelihood of these error situations.
Although currently many ADAS (advanced driver assistance) applications are based on traditional techniques using mainly computer vision algorithms, the new machine learning techniques, especially neural networks and variants of neural networks such as CNNs (convolutional neural networks) or RCNNs (region-based convolutional neural networks), are getting more and more important. In particular, RCNNs processing camera information are regarded as state-of-the-art systems for detecting, classifying and localizing dynamic and static road objects. The quality of the detection, classification and localization of objects heavily depends on many different factors, such as the underlying neural network structure or the training data used for training the parameters of the neural network. The training is a very time-consuming process which takes place offline on big servers and which requires labeled training data. Labeled training data consists of both the sensor data, e.g. camera image, and classification and localization information, e.g. bounding boxes around cars or pedestrians. After the training is completed, the neural network consisting of code and configuration data is then deployed to the HAD unit in the car. The neural network in the car allows for detection, classification and localization of static and dynamic road users from camera image streams in real time.
Functional safety is the part of the overall safety of a system or piece of equipment that depends on the system or equipment operating correctly in response to its inputs, including the safe management of likely operator errors, hardware failures and environmental changes. Titled “Road vehicles—Functional safety”, ISO 26262 is an international standard for functional safety of electrical and/or electronic systems in production automobiles defined by the International Organization for Standardization (ISO) in 2011. It provides an automotive-specific, risk-based approach for determining risk classes (Automotive Safety Integrity Levels, ASILs). ASIL classifications are used within ISO 26262 to express the level of risk reduction required to prevent a specific hazard, with ASIL D representing the highest and ASIL A the lowest. In order to reach ASIL D within a system, it is possible to combine lower ASIL components and compare their results with a plausibility voter.
There exists a need for a system to accurately correlate sensor data in a vehicle and to generate a correlated model of the objects existing in space around the vehicle, and particularly to do so using uncertain information.